-
Notifications
You must be signed in to change notification settings - Fork 10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add Bivariate Extreme Value Copulas #208
Conversation
Santymax98
commented
Jul 25, 2024
new file: docs/src/extremevalue/generalities.md modified: src/Copulas.jl new file: src/ExtremeValue.jl new file: src/ExtremeValueCopulas/AsymGalambosCopula.jl new file: src/ExtremeValueCopulas/AsymLogCopula.jl new file: src/ExtremeValueCopulas/AsymMixedCopula.jl new file: src/ExtremeValueCopulas/BC2Copula.jl new file: src/ExtremeValueCopulas/CuadrasAugeCopula.jl new file: src/ExtremeValueCopulas/GalambosCopula.jl new file: src/ExtremeValueCopulas/HuslerReissCopula.jl new file: src/ExtremeValueCopulas/LogCopula.jl new file: src/ExtremeValueCopulas/MOCopula.jl new file: src/ExtremeValueCopulas/MixedCopula.jl new file: src/ExtremeValueCopulas/tEVCopula.jl new file: src/UnivariateDistribution/ExtremeDist.jl new file: test/Extreme_value_test.jl
Hi dear @lrnv Could you help me find out an error that appears in tEVCopula? Maybe the error is that we don't have StatsFuns as a deps?? tEVCopula - sampling, pdf, cdf: Test Failed at /home/runner/work/Copulas.jl/Copulas.jl/test/Extreme_value_test.jl:293
Expression: e isa ArgumentError
Evaluated: UndefVarError(:StatsFuns) isa ArgumentError
Stacktrace:
[1] macro expansion
@ ~/work/Copulas.jl/Copulas.jl/test/Extreme_value_test.jl:293 [inlined]
[2] macro expansion
@ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Test/src/Test.jl:1151 [inlined]
[3] top-level scope
@ ~/work/Copulas.jl/Copulas.jl/test/Extreme_value_test.jl:262 The other errors are something related to test/margins_uniformity.jl | 1251 11 1262 I think they have nothing to do with Extreme Value Copulas |
I will take a look tonight -- stuck at a airport. The margins_uniformity checks that every copula has uniform margins, and if your copula do not pass the test then yes it's an issue :) |
You are travelling? for work or vacation? hahaha a well deserved rest.
It's strange because before I did an exhaustive review of the graphics, the
marginalia and some comparisons with other codes. The algorithm follows
exactly what is described in the article, I think we can check it when you
are free.
El El jue, 25 jul. 2024 a la(s) 13:26, Oskar Laverny <
***@***.***> escribió:
… I will take a look tonight -- stuck at a airport. The margins_uniformity
checks that every copula has uniform margins, and if your copula do not
pass the test then yes it's an issue :)
—
Reply to this email directly, view it on GitHub
<#208 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/A7PTWNSB3PLUSVHZAG5DB7DZOE7MXAVCNFSM6AAAAABLPBR3F6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENJRGE2DIOJWG4>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
So, Now it yells because some of them do not produce marignals uniform enough. We have to investigate why -- maybe the way i'm checking uniformity is not right ! |
Yup, this replaces #205 ; The algorithm I used is the one presented in this reference |
modified: src/Copulas.jl modified: src/ExtremeValueCopulas/tEVCopula.jl modified: src/UnivariateDistribution/ExtremeDist.jl
So I temporarily removed the KS test so that we can concentrate on the rest for the moment. I have weird behaviors of HuslerReis (and also tEV it seems ?) on the cdf when one of the two values is 1 : julia> C = HuslerReissCopula(0.1)
HuslerReissCopula{Float64}(θ=0.1)
julia> cdf(C, [1,0.7])
ERROR: DomainError with -Inf:
log was called with a negative real argument but will only return a complex result if called with a complex argument. Try log(Complex(x)).
Stacktrace:
[1] throw_complex_domainerror(f::Symbol, x::Float64)
@ Base.Math .\math.jl:33
[2] _log(x::Float64, base::Val{:ℯ}, func::Symbol)
@ Base.Math .\special\log.jl:301
[3] log
@ .\special\log.jl:267 [inlined]
[4] ℓ(H::HuslerReissCopula{Float64}, t::Vector{Float64})
@ Copulas c:\Users\lrnv\.julia\dev\Copulas\src\ExtremeValueCopulas\HuslerReissCopula.jl:45
[5] _cdf(C::HuslerReissCopula{Float64}, u::Vector{Float64})
@ Copulas c:\Users\lrnv\.julia\dev\Copulas\src\ExtremeValue.jl:25
[6] cdf(C::HuslerReissCopula{Float64}, u::Vector{Float64})
@ Copulas c:\Users\lrnv\.julia\dev\Copulas\src\Copula.jl:19
[7] top-level scope
@ REPL[83]:1
julia> And this is precisely the checks I am doing : checking that Maybe you could ammend your implementation to return the correct limit instead of failling in some cases ? maybe there are logs that need to be more carefully implemented or |
I'm going to review the edge cases when they are 1 and 0, the way to get the cdf is |
Ok i think i managed to fix some of the domain errors by replacing a Runnig the full test suite again to see what's next :) |
modified: src/ExtremeValueCopulas/LogCopula.jl
Using absolute value was a good move |
Now looks like MOCopula is failling the uniformity test of its marginals: (C, u) = (Copulas.MOCopula{Float64}(λ1=0.1, λ2=0.2, λ12=0.3), [0.30184329672330246, 1.0])
Generic tests on every copulas: Test Failed at C:\Users\lrnv\.julia\dev\Copulas\test\margins_uniformity.jl:146
Expression: ≈(cdf(C, u), val, atol = 1.0e-5)
Evaluated: 0.3835528626511967 ≈ 0.30184329672330246 (atol=1.0e-5)
Stacktrace:
[1] macro expansion
@ C:\Users\lrnv\.julia\juliaup\julia-1.10.0+0.x64.w64.mingw32\share\julia\stdlib\v1.10\Test\src\Test.jl:672 [inlined]
[2] top-level scope
@ C:\Users\lrnv\.julia\dev\Copulas\test\margins_uniformity.jl:146
(C, u) = (Copulas.MOCopula{Float64}(λ1=0.1, λ2=0.2, λ12=0.3), [0.6965294668307551, 1.0])
Generic tests on every copulas: Test Failed at C:\Users\lrnv\.julia\dev\Copulas\test\margins_uniformity.jl:146
Expression: ≈(cdf(C, u), val, atol = 1.0e-5)
Evaluated: 0.7487754473247006 ≈ 0.6965294668307551 (atol=1.0e-5)
Stacktrace:
[1] macro expansion
@ C:\Users\lrnv\.julia\juliaup\julia-1.10.0+0.x64.w64.mingw32\share\julia\stdlib\v1.10\Test\src\Test.jl:672 [inlined]
[2] top-level scope
@ C:\Users\lrnv\.julia\dev\Copulas\test\margins_uniformity.jl:146
(C, u) = (Copulas.MOCopula{Float64}(λ1=0.1, λ2=0.2, λ12=0.3), [0.6770337002357543, 1.0])
Generic tests on every copulas: Test Failed at C:\Users\lrnv\.julia\dev\Copulas\test\margins_uniformity.jl:146
Expression: ≈(cdf(C, u), val, atol = 1.0e-5)
Evaluated: 0.7319614847835554 ≈ 0.6770337002357543 (atol=1.0e-5)
Stacktrace:
[1] macro expansion
@ C:\Users\lrnv\.julia\juliaup\julia-1.10.0+0.x64.w64.mingw32\share\julia\stdlib\v1.10\Test\src\Test.jl:672 [inlined]
[2] top-level scope
@ C:\Users\lrnv\.julia\dev\Copulas\test\margins_uniformity.jl:146
(C, u) = (Copulas.MOCopula{Float64}(λ1=0.1, λ2=0.2, λ12=0.3), [0.5793654774234129, 1.0])
Generic tests on every copulas: Test Failed at C:\Users\lrnv\.julia\dev\Copulas\test\margins_uniformity.jl:146
Expression: ≈(cdf(C, u), val, atol = 1.0e-5)
Evaluated: 0.6461927642785321 ≈ 0.5793654774234129 (atol=1.0e-5)
Stacktrace:
[1] macro expansion
@ C:\Users\lrnv\.julia\juliaup\julia-1.10.0+0.x64.w64.mingw32\share\julia\stdlib\v1.10\Test\src\Test.jl:672 [inlined]
[2] top-level scope
@ C:\Users\lrnv\.julia\dev\Copulas\test\margins_uniformity.jl:146
(C, u) = (Copulas.MOCopula{Float64}(λ1=0.1, λ2=0.2, λ12=0.3), [0.6629483465032526, 1.0])
Generic tests on every copulas: Test Failed at C:\Users\lrnv\.julia\dev\Copulas\test\margins_uniformity.jl:146
Expression: ≈(cdf(C, u), val, atol = 1.0e-5)
Evaluated: 0.7197534469127619 ≈ 0.6629483465032526 (atol=1.0e-5)
Stacktrace:
[1] macro expansion
@ C:\Users\lrnv\.julia\juliaup\julia-1.10.0+0.x64.w64.mingw32\share\julia\stdlib\v1.10\Test\src\Test.jl:672 [inlined]
[2] top-level scope
@ C:\Users\lrnv\.julia\dev\Copulas\test\margins_uniformity.jl:146
(C, u) = (Copulas.MOCopula{Float64}(λ1=0.1, λ2=0.2, λ12=0.3), [0.30184329672330246, 1.0])
Generic tests on every copulas: Test Failed at C:\Users\lrnv\.julia\dev\Copulas\test\margins_uniformity.jl:146
Expression: ≈(cdf(C, u), val, atol = 1.0e-5)
Evaluated: 0.3835528626511967 ≈ 0.30184329672330246 (atol=1.0e-5)
Stacktrace:
[1] macro expansion
@ C:\Users\lrnv\.julia\juliaup\julia-1.10.0+0.x64.w64.mingw32\share\julia\stdlib\v1.10\Test\src\Test.jl:672 [inlined]
[2] top-level scope
@ C:\Users\lrnv\.julia\dev\Copulas\test\margins_uniformity.jl:146
(C, u) = (Copulas.MOCopula{Float64}(λ1=0.1, λ2=0.2, λ12=0.3), [0.6965294668307551, 1.0])
Generic tests on every copulas: Test Failed at C:\Users\lrnv\.julia\dev\Copulas\test\margins_uniformity.jl:146
Expression: ≈(cdf(C, u), val, atol = 1.0e-5)
Evaluated: 0.7487754473247006 ≈ 0.6965294668307551 (atol=1.0e-5)
Stacktrace:
[1] macro expansion
@ C:\Users\lrnv\.julia\juliaup\julia-1.10.0+0.x64.w64.mingw32\share\julia\stdlib\v1.10\Test\src\Test.jl:672 [inlined]
[2] top-level scope
@ C:\Users\lrnv\.julia\dev\Copulas\test\margins_uniformity.jl:146
(C, u) = (Copulas.MOCopula{Float64}(λ1=0.1, λ2=0.2, λ12=0.3), [0.6770337002357543, 1.0])
Generic tests on every copulas: Test Failed at C:\Users\lrnv\.julia\dev\Copulas\test\margins_uniformity.jl:146
Expression: ≈(cdf(C, u), val, atol = 1.0e-5)
Evaluated: 0.7319614847835554 ≈ 0.6770337002357543 (atol=1.0e-5)
Stacktrace:
[1] macro expansion
@ C:\Users\lrnv\.julia\juliaup\julia-1.10.0+0.x64.w64.mingw32\share\julia\stdlib\v1.10\Test\src\Test.jl:672 [inlined]
[2] top-level scope
@ C:\Users\lrnv\.julia\dev\Copulas\test\margins_uniformity.jl:146
(C, u) = (Copulas.MOCopula{Float64}(λ1=0.1, λ2=0.2, λ12=0.3), [0.5793654774234129, 1.0])
Generic tests on every copulas: Test Failed at C:\Users\lrnv\.julia\dev\Copulas\test\margins_uniformity.jl:146
Expression: ≈(cdf(C, u), val, atol = 1.0e-5)
Evaluated: 0.6461927642785321 ≈ 0.5793654774234129 (atol=1.0e-5)
Stacktrace:
[1] macro expansion
@ C:\Users\lrnv\.julia\juliaup\julia-1.10.0+0.x64.w64.mingw32\share\julia\stdlib\v1.10\Test\src\Test.jl:672 [inlined]
[2] top-level scope
@ C:\Users\lrnv\.julia\dev\Copulas\test\margins_uniformity.jl:146
(C, u) = (Copulas.MOCopula{Float64}(λ1=0.1, λ2=0.2, λ12=0.3), [0.6629483465032526, 1.0])
Generic tests on every copulas: Test Failed at C:\Users\lrnv\.julia\dev\Copulas\test\margins_uniformity.jl:146
Expression: ≈(cdf(C, u), val, atol = 1.0e-5)
Evaluated: 0.7197534469127619 ≈ 0.6629483465032526 (atol=1.0e-5) Basically you can verify it by runnig: C = MOCopula(0.1,0.2,0.3)
u = [1, rand()]
v = [rand(), 1]
(cdf(C,u), u[2]) # should be equal
(cdf(C,v), v[1]) # should be equal Moreover, it looks like it is OK for [1, rand()] but failling for [rand(), 1]. Maybe you need to check the implementation of this cdf ? |
using Distributions
using HypothesisTests
using TwinCopulas
function test_uniformity(data)
n, m = size(data)
results = []
for i in 1:n
sample = data[i, :]
ks_test = ApproximateOneSampleKSTest(sample, Uniform(0, 1))
p_value = pvalue(ks_test)
push!(results, (i, p_value))
end
return results
end
cops = (AsymGalambosCopula(0.1, [0.2,0.6]),
AsymLogCopula(1.2, [0.3,0.6]),
BC2Copula(0.7,0.3),
CuadrasAugeCopula(0.1),
HuslerReissCopula(0.1),
MixedCopula(0.2),
MOCopula(0.1,0.2,0.3),
)
for (i, C) in enumerate(cops)
println("\nProbando copula $i: $(typeof(C))")
data = rand(C, 1000) # Genera datos a partir de la copula
results = test_uniformity(data)
# Mostrar los resultados
for (marginal, p_value) in results
println("Marginal $marginal: p-value = $p_value")
if p_value < 0.05
println("Rechazamos la hipótesis nula de que la marginal $marginal proviene de una distribución uniforme.")
else
println("No podemos rechazar la hipótesis nula de que la marginal $marginal proviene de una distribución uniforme.")
end
end
end In my case I used this, and it checks uniformity for all, I have problems with Galambos and logistic (which should be equal to gumbel) |
The problem with this test is that it might just fail out of bad luck (that is the definition of a 5% p-value : in 5% of the cases it'll fail under H0)... |
You have missing references in the .bib : Joe1990, Ghoudi1998. can you add them ? |
You are right, I also noticed the bug in MOCopula, we could remove the function |
I'm terrible at placing citations, however I think I can look them up and start completing each one of them. I think I only put the names of the references without links, right, or .bib, right? |
The abstract type |
Yes you wrte the citations corectly, just forgot to add them to the .bib file |
If it is enough, yes you can try that |
Hi Oskar, sorry for not responding sooner, I'm a little sick but I'll start
going over each aspect to find the solution.
El El lun, 29 jul. 2024 a la(s) 3:49, Oskar Laverny <
***@***.***> escribió:
… @Santymax98 <https://github.com/Santymax98> Ok i put a bit of an effort
to fix the references. On top of the two or three coments that are still
unresolved in this thread, there are still a few issues:
- MOCopula still failling.
- Docstring for the abstract type ExtremeValueCopula is missing
- The documentation is not very clear about references for the
sampling algorithm, neither on which one of the two is used by the code.
I am now waiting for your input. Once you have these few things figured
out, we can merge this :)
—
Reply to this email directly, view it on GitHub
<#208 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/A7PTWNUWJRUFULG3QGOOB33ZOX6YXAVCNFSM6AAAAABLPBR3F6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENJVGM3DAMJSHE>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Sorry to hear that, hope you'll get better soon ! :/ Yeah take your time, there is no rush on my side. I also see that there are one or two more copulas in your twincopulas that could me migrated to here. Also statistics such as Blomqvist \beta & others could be migrated too if you want, with potentail restrictions to Once all of that is done, I could help you design dynamic stuff on top of the class system that is here if you want / need ! Let's stay in touch. ++ |
Yes, first I wanted to leave extreme value domes in good condition, then we
can see the other copulas, in general they have the same structure, I think
they have good implementations
El El lun, 29 jul. 2024 a la(s) 12:06, Oskar Laverny <
***@***.***> escribió:
… Yeah take your time, there is no rush on my side. I also see that there
are one or two more copulas in your twincopulas version that could me
migrated to here.
Once all of that is done, I could help you design dynamic stuff on top of
the class system that is here if you want / need !
Let's stay in touch.
++
Oskar
—
Reply to this email directly, view it on GitHub
<#208 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/A7PTWNUMP224G2WV34PBBW3ZOZZC7AVCNFSM6AAAAABLPBR3F6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENJWGQ3DAMZUG4>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #208 +/- ##
==========================================
+ Coverage 83.42% 86.32% +2.90%
==========================================
Files 29 44 +15
Lines 748 1170 +422
==========================================
+ Hits 624 1010 +386
- Misses 124 160 +36 ☔ View full report in Codecov by Sentry. |
Hello Oskar, I added the CDF directly, it appears that all the tests passed... would I need to start working with the documentation? In relation to Blomqvist |
Yes beta and gamma would go in another PR that makes sense. On this one you can correct the docs and we:ll be alright. GG for MOCopula ! |
II write it here as i think about it now: you could add up (in this PR) a line in the comparison table on the readme about bivariate extreme value copulas |
I had the same idea 🤣
El El mié, 31 jul. 2024 a la(s) 2:12, Oskar Laverny <
***@***.***> escribió:
… II write it here as i think about it now: you could add up (in this PR) a
line in the comparison table on the readme about bivariate extreme value
copulas
—
Reply to this email directly, view it on GitHub
<#208 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/A7PTWNXQBTTXA2WIMJ2BEMTZPCE7XAVCNFSM6AAAAABLPBR3F6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENJZHAZTCOBVHA>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
modified: docs/src/assets/references.bib modified: docs/src/extremevalue/generalities.md modified: src/ExtremeValueCopula.jl
The erros "Invalid sequence" seems to mean that you did not format the math correctly in the docstring : take a look at other docstrings (e.g. archimedeans) to see how it should be done. |
The output on Julia 1.10 is way better than on julia 1.6, it gives: ERROR: LoadError: ParseError:
# Error @ /home/runner/work/Copulas.jl/Copulas.jl/src/ExtremeValueCopula.jl:26:118
In this way, in order to define a bivariate copula of extreme values, it is only necessary to introduce the function \( A\).
# └┘ ── invalid escape sequence
Stacktrace: |
@lrnv Do you think it's ready? Or do we need to add something else? |
I will review it tomorrow before merging but I think it's ok like that |
@Santymax98 see #210 there is an issue in the docs |